This section contains some basic FAQ and tips. It’s here at the top so that if you get stuck or confused, you can easily find it.
help(functionname) or ?functionname or searching for the function by name in the Help tab on the right. Function arguments have names, but names can be omitted if using them in their intended order; they can be looked up in the help files.Tools -> Global Options -> R Markdown -> untick “Show plots inline…”
This indicates the package is not loaded. Use the relevant library() command to load the package that includes the missing function. There are library("package") calls in the beginning of each section that requires them. You really need to load a package once per session, but they are there anyway to keep the script modular for easier revisiting. In general, it’s better practice to have library() calls in the head of the script file.
Either the package is not installed, or you misspelled its name. You should have installed the necessary packages before the start of the workshop. If you did not (indicating by library() giving you a “package not found” error), then here are the relevant installation commands.
# This is a code block, distinguishable by the gray shaded background.
# This is a line of code:
print( "Hello! Put your text cursor on this line (click on the line). Anywhere on the line. Now press CTRL+ENTER (PC) or CMD+ENTER (Mac). Just do it." )
# The command above, when executed (what you just did), printed the text in the console below. Also, this here is a comment. Commented parts of the script (anything after a # ) are not executed. This R Markdown file has both code blocks (gray background) and regular text (white background).
(Also, if you’ve been scrolling left and right in the script window to read the code, turn on text wrapping ASAP: on the menu bar above, go to Tools -> Global Options -> Code (tab on the left) -> tick “Soft-wrap R source files”)
So, print() is a function. Most functions look something like this:
myfunction(inputs, parameters)All the inputs to the function go inside the ( ) brackets, separated by commas. In the above case, the text is the input to the print() function. All text, or “strings”, must be within quotes. Most functions have some output. Note that commands may be nested; in this case, the innermost are evaluated first:
function2( function1(do, something), parameters_for_function1 )Don’t worry if that’s all a bit confusing for now. Let’s try another function, sum():
sum(1,10) # cursor on the line, press CTRL+ENTER (or CMD+ENTER on Mac)
# You should see the output (sum of 1 and 10) in the console.
# Important: you can always get help for a function and check its input parameters by executing
help(sum) # put the name of any function in the brackets
# ...or by searching for the function by name in the Help tab on the right.
# Exercise. You can also write commands directly in the console, and executing them with ENTER. Try some more simple maths - math in R can also be written using regular math symbols (which are really also functions). Write 2*3+1 in the console below, and press ENTER.
# Let's plot something. The command for plotting is, surprisingly, plot().
# It (often) automatically adopts to data type (you'll see how soon enough).
plot(42, main = "The greatest plot in the world") # execute the command; a plot should appear on the right.
# OK, that was not very exciting. But notice that a function can have multiple inputs, or arguments. In this case, the first argument is the data (a vector of length one), and the second is 'main', which specifies the main title of the plot.
# You can make to plot bigger by pressing the 'Zoom' button above the plot panel on the right.
# Let's create some data to play with. We'll use the sample() command, which creates random numbers from a predifined sample. Basically it's like rolling a dice some n times, and recording the results.
sample(x = 1:6, size = 50, replace = T) # execute this; its output is 50 numbers
# Most functions follow this pattern: there's input(s), something is done to the input, and then there's an output. If an output is not assigned to some object, it usually just gets printed in the console. It would be easier to work with data, if we saved it in an object. For this, we need to learn assignement, which in R works using the equals = symbol (or the <-, but let's stick with = for simplicity).
dice = sample(x = 1:6, size = 50, replace = T) # what it means: xdata is the name of a (new) object, the equals sign (=) signifies assignement, with the object on the left and the data on the right. In this case, the data is the output of the sample() function. Instead of printing in the console, the output is assigned to the object.
dice # execute to inspect: calling an object usually prints its contents into the console below.
# Let's plot:
hist(dice, breaks=20, main="Frequency of dice values") # plots a histogram (distribution of values)
plot(dice) # plots data as it is ordered in the object
xmean = mean(dice) # calculate the mean of the 50 dice throws
abline(h = xmean, lwd=3) # plot the mean as a horizontal line
# Exercise: compare this plot with your neighbor. Do they look the same? Why/why not?
# Exercise: use the sample() function to simulate 25 throws of an 8-sided DnD dice.
We’ll need some data for the examples. R has a range of functions for importing various types of data. We will pull data from the SCYOSA API (https://scotssyntaxatlas.ac.uk/api/v1/), which offers json and csv formats.
# Let's get all the locations.
# read.csv can read from local disk as well as the net
# the "skip" argument skips lines - the first line in the data from the API is a description
locs = read.csv("http://scotssyntaxatlas.ac.uk/api/v1/csv/locations", skip=1)
locs[[7]] = NULL # remove column 7, which we won't need for now (there's a lot of data in it)
head(locs) # use head() to have a look at the first lines of the new dataframe
## display_town display_lat display_lng num qids id
## 1 Aberdeen 57.14972 -2.094278 4 652,653,654,655 1
## 2 Aboyne 57.07667 -2.780312 4 590,591,592,593 2
## 3 Airdrie 55.86627 -3.962566 4 131,134,658,667 3
## 4 Alloa 56.11407 -3.791896 4 588,589,602,603 4
## 5 Alloway 55.42835 -4.634692 4 336,337,338,339 5
## 6 Annan 54.99025 -3.259773 4 414,415,416,417 6
summary(locs) # use summary to get an overview of the columns
## display_town display_lat display_lng num
## Aberdeen: 1 Min. :54.73 Min. :-7.2820 Min. :2.000
## Aboyne : 1 1st Qu.:55.79 1st Qu.:-4.4012 1st Qu.:4.000
## Airdrie : 1 Median :56.11 Median :-3.5368 Median :4.000
## Alloa : 1 Mean :56.52 Mean :-3.7029 Mean :3.829
## Alloway : 1 3rd Qu.:57.34 3rd Qu.:-2.9986 3rd Qu.:4.000
## Annan : 1 Max. :60.50 Max. :-0.9818 Max. :8.000
## (Other) :140
## qids id
## 105,106,107,108: 1 Min. : 1.00
## 113,116,118,129: 1 1st Qu.: 38.25
## 114,115,117,121: 1 Median : 74.50
## 119,120,124,125: 1 Mean : 74.43
## 122,123,130,142: 1 3rd Qu.:110.75
## 131,134,658,667: 1 Max. :147.00
## (Other) :140
# What's in there?
# Let's get the attributes overview data
att = read.csv("http://scotssyntaxatlas.ac.uk/api/v1/csv/attributes", skip=1)
# Investigate the new object.
# Let's have a look at the column which shorthands for the attributes: att$pname
att[182:183,] # maybe that's interesting?
## pid code pname pdesc atname
## 182 17 m1 PLUR Een
## 183 17 m2 PLUR Sheen
## atdescription aid soundclips
## 182 He got something in his een! 117
## 183 You have to take off your sheen at the door. 118
# Download the relevant rating data
a117 = read.csv("http://scotssyntaxatlas.ac.uk/api/v1/csv/attribute/117", skip=1)
a118 = read.csv("http://scotssyntaxatlas.ac.uk/api/v1/csv/attribute/118", skip=1)
# We'll be using the a177 for the first examples
dim(a117) # quite a bit of data!
## [1] 547 13
We’ll be using an alternative plotting package to base graphics, ggplot2. It uses a different approach to plotting, and a slightly different syntax. It also comes with default colors and aesthetics which many people find nicer than those of the base plot(). A particularly useful feature of ggplot2 is its extendability (or rather the fact people are eager to extend it), with an ever-growing list of addon-packages on CRAN offering a wider selection of themes and all sorts of visualization types.
library(ggplot2) # load the package
library(ggbeeswarm) # a little addon
# ratings for "He got something in his een!"
ggplot(a117, aes(x=rating)) +
geom_histogram()
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
# compare old and young respondents
ggplot(a117, aes(x=rating, fill=agegroup)) +
geom_bar( width=0.7, position = "dodge")
# We could also see where the respondents come from, in terms of north and south:
# (there are probably better ways than latitude to control for areal variation; this is mostly just to explore ggplot a bit more)
ggplot(a117, aes(y=display_lat, x=0)) +
geom_boxplot()
# Exercise: boxplots are nice and all, but we can try other options as well. Try replacing geom_boxplot() above with geom_violin() or geom_beeswarm() (the latter comes from the ggbeeswarm addon package). You can also add geom_point(shape=95, size=3) to geom_violin() for a rug plot effect.
# do the ratings for the "plurals" attributes correlate?
plurals = merge(a117, a118, by.x=1, by.y=1, suffixes = c(117,118)) # merge the two together into a single dataframe, so we can compare ratings
dim(plurals)
## [1] 541 25
colnames(plurals)
## [1] "qid" "display_town117" "display_lat117"
## [4] "display_lng117" "cid117" "rating117"
## [7] "spurious117" "comments117" "ra_comments117"
## [10] "isfw117" "atname117" "atdesc117"
## [13] "agegroup117" "display_town118" "display_lat118"
## [16] "display_lng118" "cid118" "rating118"
## [19] "spurious118" "comments118" "ra_comments118"
## [22] "isfw118" "atname118" "atdesc118"
## [25] "agegroup118"
ggplot(plurals, aes(x=rating117, y=rating118)) +
geom_point(alpha=0.5)
# Hmm... this is not a very useful plot though is it now? Could we make it better?
Exercises: make this plot useful
ccolor=agegroup117to the aes() above.+ xlab(levels(plurals$atdesc117)[1]) and + ylab(levels(plurals$atdesc117)[1])scale_colour_brewer(palette = "Dark2")theme_minimal()- start typing theme_ and see what RStudio’s autocomplete offers.# We could also see if variation correlates with longitude or latitude.
# (again, probably there are better ways to actually quantify areal variation)
# Let's keep the old and the young separate though.
library(cowplot) # package to arrange multiple ggplots
##
## Attaching package: 'cowplot'
## The following object is masked from 'package:ggplot2':
##
## ggsave
plot_grid(nrow=2,
ggplot(a117, aes(x=rating, y=display_lng, color=agegroup)) +
geom_point(alpha=0.5, position = position_jitter(width=0.1)) +
geom_smooth(method="lm") +
facet_wrap("agegroup") + theme_bw() +
ylab("<-west east->") +
xlab('"He got something in his een!"')
,
ggplot(a117, aes(x=rating, y=display_lat, color=agegroup)) +
geom_point(alpha=0.5, position = position_jitter(width=0.1)) +
geom_smooth(method="lm") +
facet_wrap("agegroup") + theme_bw() +
ylab("<-south north->")
)
# Exercise: discuss the plot with your neighbor.
# We can also visualize categorical (countable) data:
ggplot(a117, aes(x=agegroup)) +
geom_bar()
# Exercises:
# What's in this plot?
# Change the x value to display_town to see the number of participants per location. Add + coord_flip() to make it a bit more readable.
How about putting these values on actual maps though?
library(ggplot2)
# We could have a look at all the locations covered by the questionnaire (for this question), since we have the coordinates:
ggplot(locs, aes(x=display_lng, y=display_lat, label=display_town)) +
geom_text(size=2)
# We could also plot all the datapoints we saw earlier according to their location data.
# Jittering makes the points easier to see.
ggplot(a117, aes(x=display_lng, y=display_lat, shape=agegroup, color=rating)) +
geom_point( alpha=0.7, position=position_jitter(width = 0.2, height = 0))
# Exercise: add + scale_color_viridis_c() for a more distinctive scale, and add the alpha=0.7 parameter to geom_point to make the points a bit transparent.
But it would be useful to see the actual map as well.
library(rworldmap)
## Loading required package: sp
## ### Welcome to rworldmap ###
## For a short introduction type : vignette('rworldmap')
library(magrittr)
# We'll fetch a generic map from the rworldmap package
data("countryExData", envir = environment(), package = "rworldmap")
uk <- joinCountryData2Map(countryExData, joinCode = "ISO3", nameJoinColumn = "ISO3V10", mapResolution = "low") %>% fortify(mymap) %>% subset(id=="United Kingdom")
## 149 codes from your data successfully matched countries in the map
## 0 codes from your data failed to match with a country code in the map
## 95 codes from the map weren't represented in your data
## Regions defined for each Polygons
# Let's just plot the map first. coord_fixed can be used to limit the map to Scotland.
ggplot() +
geom_polygon(data=uk, aes(long, lat, group = group), inherit.aes = F) +
coord_fixed(xlim=c(-7.2,-1), ylim = c(55,60.7))
# Exercise: specify color="black", fill="white" for geom_polygon for a nicer map.
# add + theme_bw() for a more suitable theme.
# remove the useless axis labels with: + theme(axis.title = element_blank())
# Exercise: we could now put the points on the map. Use the map code you just completed, and use the code from earlier that plotted (just the) points by longitude and latitude.
# Make a new plot here:
# Hint. If this is hard: you'll want to copy the ggplot with the geom_polygon from above, but specify the following in the main ggplot() data=a117, aes(y=display_lat, x=display_lng). Also add + geom_point() to show the points (after geom_polygon, so the points show up on top)
But wouldn’t it be super nice if we could also hover over the points on the map and get more info on each datapoint, and zoom in to interesting areas?
library(plotly)
##
## Attaching package: 'plotly'
## The following object is masked from 'package:ggplot2':
##
## last_plot
## The following object is masked from 'package:stats':
##
## filter
## The following object is masked from 'package:graphics':
##
## layout
# plotly is a package for making interactive plots. You can construct your plots from scratch using plot_ly(), but you can also convert existing ggplots into interactive plotly plots.
# You can also make animations, 3D plots, and app-like plots with multiple views using plotly.
# First let's make a basic map. Maybe a different theme this time, just for fun.
# Save it as an object
# [this is a placeholder; you'll insert some code here later]
g = ggplot(a117, aes(x=display_lng, y=display_lat, color=rating)) +
geom_polygon(data=uk, aes(long, lat, group = group), inherit.aes = F,fill="black") +
coord_fixed(xlim=c(-7.2,-1), ylim = c(55,60.7)) +
geom_point(alpha=0.8, size=0.7, position=position_jitter(width = 0.1, height = 0)) +
scale_color_gradient2(low="#FC8D59",midpoint=3, mid="#FFFFBF", high="#91BFDB",) +
theme_dark() +
ggtitle("He got something in his een!") +
theme(plot.background = element_rect(fill = "darkgray"),
legend.background = element_rect(fill = "darkgray"),
axis.title = element_blank())
g # call the object to have a look
ggplotly(g) # make it interactive; need the plotly package.
# Zooming by default works by click-dragging a rectangle over the plot; we could also enable mouse-wheel (Mac: double-finger) zooming:
ggplotly(g) %>% config(scrollZoom = TRUE)
# Currently the hover labels display only the latitude and longitude info, and the rating. We could add more to that. See that [ placeholder ] line above? Move this line up there:
a117$labels = paste(a117$display_town, a117$agegroup, sep = "<br>")
# Also add the following parameter to the aes() in the main ggplot() call: text=labels
# Now run the ggplot again, and then ggplotly()
library(quanteda)
## Package version: 1.4.3
## Parallel computing: 2 of 4 threads used.
## See https://quanteda.io for tutorials and examples.
##
## Attaching package: 'quanteda'
## The following object is masked from 'package:utils':
##
## View
sent = att$atdescription # the questionnaire sentences
# Quick text processing: remove comments, only keep actual sentences, using regex:
sent = gsub('^[^"]+\"([^"]+)".*', "\\1", sent)
# (this is a quick hack, not perfect)
# More text processing, using quanteda: lowercase, remove stopwords, punctuation, spaces
parsed = dfm(sent,
remove = stopwords('english'),
remove_punct = TRUE, tolower = T )
parsed[1:10,1:10] # quick look at the new data structure
## Document-feature matrix of: 10 documents, 10 features (88.0% sparse).
## 10 x 10 sparse Matrix of class "dfm"
## features
## docs div like scone needs washed didnae got shoes well just
## text1 1 1 1 0 0 0 0 0 0 0
## text2 0 0 0 1 1 0 0 0 0 0
## text3 0 1 0 0 0 1 0 0 0 0
## text4 0 0 0 0 0 0 1 1 1 0
## text5 0 0 0 0 0 0 0 0 0 1
## text6 0 0 0 0 0 0 0 0 0 1
## text7 0 0 0 0 0 0 0 0 0 0
## text8 0 0 0 0 0 0 0 0 0 0
## text9 0 0 0 0 0 0 0 0 0 0
## text10 0 0 0 0 0 0 0 0 0 0
# Plot 100 most frequent words in the data:
textplot_wordcloud(parsed, min_count = 1, color=terrain.colors(100), max_words = 100)
# We could also visualize the locations; locations with more respondents are bigger:
textplot_wordcloud(dfm(as.character(a117$display_town)), max_size = 1.5, color=gray.colors(100,start = 0, end = 0.4))
It’s fairly straightforward to produce html content (websites, slides, posters, books) in R using R Markdown. We’ll need to create a new file for this part.
First let’s save a plot we made before. When producing an html file from an R Markdown rmd file, functions and objects in the current global environment cannot be accessed. That means that if you’ll need to include library() calls in the rmd, as well as load or import data. It often makes sense to deal with data processing in a separate script, save the results as an .RData file, and then just load the RData (using load("file.RData")) in the markdown file you intend to knit, instead of doing data cleaning and analysis upon every time you re-knit.
library(ggplot2)
# Here's a variation of a plot we already looked at before. Instead of saving data and including plotting code in the new file, we can also save the actual ggplot object.
g = ggplot({a117$rating = as.factor(a117$rating); a117}, aes(x=display_lng, y=display_lat, shape=agegroup, color=rating) ) +
geom_polygon(data=uk, aes(long, lat, group = group), inherit.aes = F, color="gray", fill="white") +
geom_point( alpha=0.7, position=position_jitter(width = 0.15, height = 0)) +
coord_fixed(xlim=c(-7.2,-1), ylim = c(55,60.7)) +
annotate("text", x =c(-3.1883, -4.2518, -2.0943,-1.1493), y=c(55.9533, 55.8642, 57.1497,60.1530), label=c("Edinburgh", "Glasgow", "Aberdeen", "Lerwick"), color="black",hjust="left" ) +
scale_color_brewer(palette = "Spectral") +
theme_bw() +
theme(axis.title = element_blank())
g
Save this plot using this:
save(g, file="test.RData") # this will save the plot within an RData file in your default working directory. You can see where it went by running:
getwd()
Great. Exercise time. Click on the icon with the green plus on a white background in the top left corner, choose “R Markdown…”, “Document” will be by default selected on the left, and “HTML” on the right - click “OK” (change the title and author if you want).
Now copy this code block, the entire block, starting from the next line here:
…end ending on the empty line above this.
Go to your new Rmd file, remove the template stuff below the yaml header (that ends with the ---). Now paste the code block into the new file. Make sure to change eval=FALSE to eval=TRUE so that this block will run. You can also add some text if you want. Then click “Knit” (next to the little bar of yarn icon) on the top toolbar. RStudio will ask you to save the new file, just save it in the default working directory, where the RData was saved.
Notably, we could also just save the ggplot as a static image (using ggsave()), but this example shows you how to execute R code within an R Markdown file upon knitting to html. You might also want to check out the flexdashboard package, which makes it easy to create nice dashboards (you can include hmtlwidgets like plotly), or the shiny package for making web apps that users can explore and interact with (this requires a server setup, but demos can be published on shinyapps.io for free).
A couple of examples of things that I’ve myself used R Markdown for:
library(magrittr)
# This bit introduces magrittr's pipe %>% command. It's super useful! Also some packages like plotly, dplyr, networkd3 kind of expect you to use the pipe syntax.
# The shortcut in RStudio is CTRL-SHIFT-M (or CMD-SHIFT-M).
# If you're familiar with Bash pipes: it's the same thing.
# If you're interested why the somewhat curious name: https://en.wikipedia.org/wiki/The_Treachery_of_Images
# Exercise. Try it out and discuss the results with your neighbor.
1:3
## [1] 1 2 3
sum(1:3)
## [1] 6
x=1:3
sum(x)
## [1] 6
1:3 %>% sum() # same result, and not much difference in spelling it out either
## [1] 6
1:3 %>% sum() %>% rep(times=4) # what does that do?
## [1] 6 6 6 6
# "." can be used as a placeholder if the input is not the first argument, so the above could also be spelled out as:
1:3 %>% sum(.) %>% rep(., times=4) # or
## [1] 6 6 6 6
1:3 %>% sum(.) %>% rep(., 4) # and it's the same as
## [1] 6 6 6 6
rep(sum(1:3), times=4)
## [1] 6 6 6 6
# another example:
c(1,1,1,2) %>% match(x=2, table=.) #
## [1] 4
# something longer (take it apart to see how it works):
"hello" %>% gsub(pattern="h", replacement="H", x=.) %>% paste(., "world")
## [1] "Hello world"